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Abstract. In a previous paper, we proved that over a finite field k of suffi- 
ciently large cardinality, all curves of genus at most 3 over k can be modeled 
by a bivariate Laurent polynomial that is nondegenerate with respect to its 
Newton polytope. In this paper, we prove that there are exactly two curves 
of genus at most 3 over a finite field that are not nondegenerate, one over F2 
and one over F3. Both of these curves have remarkable extremal properties 
concerning the number of rational points over various extension fields. 



Let A; be a perfect field with algebraic closure k. To a Laurent polynomial 
/ = J2(i j)GZ 2 C %3 X% V^ G fc[a; ±1 ,i/ ±1 ], we associate its Newton polytope A(/), the 
convex hull in M 2 of the points (i, j) £ Z 2 for which 7^ 0. An irreducible Laurent 
polynomial / is called nondegenerate with respect to its Newton polytope if for all 
faces t C A(/) (vertices, edges, and A(/) itself), the system of equations 

< \ fl d ^ df ^ n 

^2 

has no solution in k , where f\ T = Y^u j)ez 2 nr c ij xl y 3 ■ 

A curve C over k is called nondegenerate if it is birationally equivalent over k 
to a curve defined by a Laurent polynomial / S k\x , y ] that is nondegenerate 
with respect to its Newton polytope. For such a curve, a vast amount of geometric 
information is encoded in the combinatorics of A(/). For example, the (geometric) 
genus of C is equal to the number lattice points (points in Z 2 ) lying in the interior of 
A(/). Owing to this connection, nondegenerate curves have become popular objects 
of study in explicit algebraic geometry. (See e.g. Batyrev pQ and the introduction 
in our preceding work [5] for further background and discussion.) 

In a previous paper [5] , we gave a partial answer to the natural question: Which 
curves are nondegenerate? 

Theorem. Let C be a curve of genus g over k. Suppose that one of these conditions 
holds: 

(i) 9 = 0; 

(ii) g = 1 and C(k) £ 0; 

(iii) <7 = 2, 3, and either 17 < ^fc < 00, or #k — 00 and C{k) ^ 0; 

(iv) g — 4 and k = k. 

Then C is nondegenerate. 

If g ^ 5, then the locus A4g d of nondegenerate curves inside the coarse moduli 
space of curves of genus g satisfies dimA4" d = 2g + 1, except for g — 7 where 
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dimA'J" 11 = 16. In particular, a generic curve of genus g is nondegenerate if and 
only if g < 4. 

Throughout the rest of this article, we assume that k is a finite field, and we 
consider the cases excluded in condition (iii) above by the condition that #fc > 17. 
Based on a number of preliminary experiments, we guessed [3 Remark 7.2] that this 
condition is superfluous. In truth we have the following theorem, which constitutes 
the main result of this paper. 

Theorem. Let C be a curve of genus g < 3 over a finite field k. Then C is 
nondegenerate unless k — F 2 or k — ¥3, and C is birational to 

C 2 : (x + y) 4 = (xy) 2 + xy(x + y + 1) + (x + y + l) 2 over F 2 , 
C 3 : y 3 - y — (x 2 + l) 2 over F 3 , 

respectively. 

Both C 2 and C3 have genus 3. In particular, all curves of genus 2 are nondegenerate. 

Intriguingly, C 2 and C3 have other remarkable properties: they obtain an ex- 
tremal number of rational points over certain extension fields of F2 and F3, respec- 
tively. 

The paper is organized into four sections. In Sections HHH we refine the bound 
on #k which guarantees that a curve of genus 2 or 3 over k is nondegenerate. 
In Section [3l we perform an exhaustive computation using the computer algebra 
system Magma [3] to reduce the bound further. At the same time, we search the 
remaining finite fields F 2 and F 3 for curves that are not nondegenerate. We conclude 
by discussing the extremal properties of the two resulting curves in Section [5] 

1. Refining the bound for hyperelliptic curves 

If char k is odd, then any hyperelliptic curve over k is easily seen to be nondegen- 
erate. Indeed, it is well-known that a hyperelliptic curve of genus g is birationally 
equivalent over k to an affine curve of the form y 2 — p(x), where p(x) £ k[x] is a 
squarefree polynomial of degree 2g + 1 or 2g + 2. Then directly from the definition 
(*), one sees that the polynomial f(x, y) — y 2 —p(x) is nondegenerate with respect 
to its Newton polytope. 

If instead charfc = 2, then a hyperelliptic curve of genus g has an affine model 
of the more general form 

(1.1) y 2 + r(x)y = p(x) 

with r(x) € k[x] of degree at most g + 1, and p(x) € k[x) of degree at most 2g + 2, 
and at least 2g + 1 if degr(x) < g + 1 (see Enge [7] Theorem 7]). Moreover, such 
a model will not have any singularities in the affine plane; however, this condition 
is not enough to ensure that the defining polynomial f(x, y) = y 2 + r(x)y + p(x) is 
nondegenerate with respect to its Newton polytope. 

Remark 1.2. There is a small erratum in our previous paper [5J Section 5]. We 
write that one can always take 2 degr(x) < degp(x) and degp(x) € {2g + l,2g + 2} 
in (jl.ip . This might however fail if k = F 2 and the hyperelliptic curve C has 
the property that the degree 2 morphism tt : C — > P 1 is completely split over fc, 
i.e., there are two distinct points in C(k) above each point 0, l,oo e P 1 (fc). This 
erratum has no effect on any further statement in the paper [5]. 

The main result of this section is as follows. 
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Proposition 1.3. Let C be a hyperelliptic curve of geometric genus g > 2 over a 
finite field k. If #k is odd or ff=k > g + 4, then C is nondegenerate. 

Proof. Let #fc = q. By the above, we may assume that q > 8 is even and that C is 
given by an equation of type (jl.ip . Let f{x, y) = y 2 + r(x)y + p(x). 

First, we claim that after applying a birational transformation we may assume 
that r(x) is a polynomial of degree g + 1 with nonzero constant term. Since q > 
g + 4 > g + 1, there is an a £ A: such that r(x — a) has nonzero constant term, so 
replacing x <— x — a we may assume r(x) has nonzero constant term. Then the 
transformed polynomial 



to the exponent vectors of f(x,y), has the property that degr'(a;) = g+ 1. Making 
another substitution x <— x — b then completes the argument. 

Then using the definition (*), a short case-by-case analysis of the possible Newton 
polytopes shows that if p(x) is squarefree, then f(x, y) — y 2 + r(x)y + p(x) is 
nondegenerate with respect to its Newton polytope. For each t{x) G k[x] of degree 
at most 5+1, consider the change of variables y <— y + t(x); then under this 
transformation we have p{x) *— Pt{%) = p(x)+r(x)t(x)+t(x) 2 andr(x) is unchanged 
(since char/c = 2). We use a sieving argument to show that there exists a choice of 
t(x) such that pt{x) is squarefree. Note we have q 9+2 choices for t(x). 

Suppose that p t is not squarefree. Then pt(x) is divisible by the square of a 
monic irreducible polynomial v(x) of degree m < g + 1. But note that if v 2 \ p tl 
and v 2 | pt 2 for two choices £i,i2, then subtracting we have 



Moreover, if v divides each of these two factors then in fact v \ r. 

We are then led to consider two cases. First, suppose that v \ r. Then either 
v 2 | (fi + t 2 ) or v 2 \(r + h+ t 2 ). Let h = [(g + l)/2j. If m = degu < h, then by 
sieving we conclude that v 2 | p t for at most 2q 9+1 ~ 2m+1 = 2q 9+2 ~ 2m values of t. 
On the other hand, if m > h then degv 2 > g + 1 so by sieving we now have v | pt 
for at most two values of t. Since the number of monic irreducible polynomials of 
degree m over k is bounded by q m /m, the number of values of t such that p t is 
divisible by v 2 with v \ r is at most 



f'(x, y) = z 29+2 /(l/:r, - y 2 + r'(x)y + p'(x) 

which corresponds to applying the applying the Z-affine map 
(X,Y)»(2g + 2-X-(g + l)Y,Y) 



v 



' 2 | (r(ti + t 3 ) + t 2 + i|) = (ti + t 2 ){r + ti+ t a ). 



g(2g^ 2 - 2 ) + ^(2 g .' 



,9+2-4 



+ 



) + 2 



/i+ 1 



+ 




.9+1 



,5+1 





(note h > 1) 
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Next, suppose that v \ r. Then in any case v \ (ti + i2), and hence there are at 
most q9+ 1 - m + 1 values of t such that v 2 \ pt- Since degr < g + 1, in the worst case 
r splits into g + 1 linear factors over fc, and we have at most (g + l)q 9+1 values of 
t for which p t is divisible by v 2 for some v \ r. 

Putting these together, we can find a value of t(x) such that pt{x) is squarefree 

if 

V .9 + 1 Q-IJ 

which holds whenever q > g + 4, since g > 2 and q > 8. □ 

For our genera of interest g — 2 and g — 3, Proposition 11.31 proves that all 
hyperelliptic curves are nondegenerate except possibly over F 2 and F4. 

2. Refining the bound for plane quartics 

In this short section, we refine the bound as in Section [1] but now for plane 
quartics. 

Lemma 2.1. Let C C P 2 be a nonsingular plane quartic over a finite field k. If 
=ffk > 7, then C is nondegenerate. 

Proof. Again analyzing the conditions of nondegeneracy [U Examples 1.5-1.6], we 
see that to prove that C is nondegenerate it suffices to find three nonconcurrent 
/c-rational lines in P 2 which are not tangent to C. The projective transformation 
which maps the three intersection points to the coordinate points (and the lines to 
the coordinate lines) realizes C as nondegenerate with respect to a Newton polytope 
of the following type: 




(A dashed line appears as a face if our transformed curve contains the corresponding 
coordinate point.) 

Write m = #C(k) and q = #fc. Since there are q 2 + q + 1 lines which are 
fc-rational in P 2 , and the number of /c-rational lines through a fixed point is q + 1 , 
it suffices to prove that C has strictly less than q 2 fc-rational tangent lines. 

We claim that the number of fc-rational tangent lines is at most m + 28. Of 
course each point of C(k) determines a tangent line. Suppose a fc-rational line is 
tangent at a point of C(k) \ C(fc); then it is also tangent at each of the Galois 
conjugates of the point, which since C is defined by a plane quartic immediately 
implies that the point is defined over a quadratic extension and that the line is a 
bitangent. By classical geometry and the theory of theta characteristics, there are 
at most 28 bitangents (see e.g. Ritzenthaler [12j Corollary 1]), and this proves the 
claim. 
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Thus if q 2 > m + 28, we can find three nonconcurrent nontangent lines. By the 
Weil bound, it is sufficient that 

q 2 >q+ 1 + 6^9 + 28 

which holds whenever q > 8. In fact, when q = 7 then m < 20 by a result of Serre 
[13] (see also Top Q3|), and so q 2 > m + 28 for all q > 7. □ 

This lemma therefore proves that all plane quartics defined over finite fields are 
nondegenerate except possibly over ¥ q with q < 5. 

3. Computational results 

From the results of the previous two sections, in order to prove our main theorem 
we perform an exhaustive computation in Magma to deal with the remaining cases: 

(la) hyperclliptic curves of genus g = 2 over F2 and F4; 

(lb) hyperelliptic curves of genus g = 3 over F2 and F4; 
(2) nonsingular quartics in P 2 over Fa, F3, F4 and F5 (genus g = 3). 
To this end, we essentially enumerated all irreducible polynomials whose Newton 
polytope is contained in 





respectively, regardless of whether they define a curve of genus g or not. For each of 
these, we checked whether the Newton polytope contained g interior lattice points, 
since by Baker's inequality 2, Theorem 4.1] an irreducible Laurent polynomial 
/ € fc[a; ±:L , y^ 1 ] defines a curve whose (geometric) genus is at most the number of 
lattice points in the interior of A(/). 

The polynomials / that passed this test were then checked for nondegeneracy 
with respect to the edges of A(/). Checking nondegeneracy with respect to the 
edges boils down to checking squarefreeness of a number of univariate polynomials 
of small degree, which can be done very efficiently. The nondegeneracy condition 
with respect to the vertices of A(/) is automatic. The nondegeneracy condition 
with respect to A(/) itself is also automatic if / defines a genus g curve (by Baker's 
inequality), so we can disregard any polynomial for which this condition is not 
satisfied. 

The polynomials / that were not nondegenerate with respect to the edges then 
saw further investigation. First, and only at this stage, we verified that in fact / 
defines a curve of genus g. Then, repeatedly, we applied a random transformation 
to / of the following form: 

(1) (x, y) (x — a,y — h(x)) for a <E k and h(x) G k[x] of degree at most g + 1 
(for hyperelliptic curves); 

(2) a projective linear transformation (for plane quartics). 

We then again checked the resulting polynomial for nondegeneracy with respect to 
the edges. Polynomials for which there were 1000 failures in a row were stored in 
a list. 

In each of the hyperelliptic curve cases the list remained empty, implying the 
following lemma. 
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Lemma 3.1. All hyperelliptic curves of genus at most 3 defined over a finite field 
are nondegenerate. 

In the plane quartic case, the list eventually contained exactly one polynomial 
for k = F 2 : 

/a : (as + y f + {xy f + xy(x + y + 1) + (x + y + l) 2 . 

We then tried all projective linear transformations in PGLa(F2) and found that, 
quite remarkably, fa is invariant under each of these transformations — the canonical 
embedding here is truly canonical! 

Over k — ¥3, we were left with a set of polynomials that turned out to be all 
projectively equivalent to the polynomial 

fa = y 3 -y-(x 2 + i) 2 . 

We exhaustively verified that none of the projectively equivalent polynomials is 
nondegenerate with respect to its Newton polytope. 

Over F4 and F5, the list remained empty. We therefore have the following propo- 
sition. 

Proposition 3.2. Over any finite field k, all curves C/k of genus at most 3 are 
nondegenerate, except if k = ¥2 and C is k-birationally equivalent to C2, or if 
k = F3 and C is k-birationally equivalent to C3. 

Proof. It remains to show that if C is a nonhyperelliptic curve of genus 3 which can 
be modeled by a nondegenerate Laurent polynomial /, then it can be modeled by 
a nondegenerate Laurent polynomial whose Newton polytope is contained in 4E, 
the convex hull of the points (0, 0), (0, 4), and (4, 0). This is true because A(/) has 
three interior lattice points which are not collinear, since C is not hyperelliptic 
Lemma 5.1]. Applying a Z-affine transformation to the exponent vectors, we may 
assume that in fact the interior lattice points of A(/) are (1, 1), (1, 2), and (2, 1). 
But then A(/) is contained in the maximal polytope with these interior lattice 
points, which is 4S [5j Lemma 10.2]. The result follows. □ 

We conclude with a remark on the total complexity of the above computation. 
Since we are only interested in curves up to birational equivalence, rather than 
simply enumerating all polynomials of a given form one could instead enumerate 
curves by their moduli. Questions of this type in low genus have been pursued by 
many authors: Cardona, Nart, and Pujolas [4] and Espinosa Garcia, Hernandez 
Encinas, and Munoz Masque [8] study genus 2; Nart and Sadornil pj] study hy- 
perelliptic curves of genus 3; and Nart and Ritzenthaler |10j study nonhyperelliptic 
curves of genus 3 over fields of even characteristic. In this paper we used a more 
naive approach since it is more transparent, easier to implement, and at the same 
time still feasible. 

We did however make use of the following speed-ups. For hyperelliptic curves of 
genus g = 3 with #k — 4, the coefficient of x s and the constant term can always 
be taken 1; for plane quartics with #fc = 4, the coefficients of x A and y A and the 
constant term can always be taken 1. Finally, for plane quartics with #=fc = 5, from 
the proof of Lemma 12.11 we may assume that there exist at least two fc-rational 
tangent lines that are only tangent over k (otherwise there exist enough nontangent 
lines to ensure nondegeneracy) ; transforming these to x- and y-axis, we may thus 
assume that f(x,0) = (ax 2 + bx + l) 2 and f(0,y) — (cy 2 + dy + l) 2 with a,b,c,d S k. 
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4. Extremal properties 

In this section, let C2 and C3 denote the complete nonsingular models of the 
curves defined as in the main theorem. 

The curve C2 can be found in many places in the existing literature. It en- 
joys some remarkable properties concerning the number #C2(F2m) of F2™-rational 
points for various values of m. First, it has no F2-rational points. However, over 
F4 and F§ it has 14 and 24 points, respectively; in both cases, this is the maximal 
number of rational points possible on a complete nonsingular genus 3 curve, and in 
each case C2 is the unique curve obtaining this bound (up to isomorphism). How- 
ever, over F32 the curve becomes pointless again! And once more, it is the unique 
curve having this property. For the details, see Elkies [6j Section 3.3]. We refer to 
work of Howe, Lauter, and Top J9j Section 4] for more on pointless curves of genus 
3. It is remarkable that this curve is also distinguished by considering conditions 
of nondegeneracy. 

In fact, C2 is a twist of the reduction modulo 2 of the Klein quartic (defined by 
the equation x 3 y+y 3 z+z 3 x — 0), which has more extremal properties. For instance, 
Elkies 6, Section 3.3] has shown that the Klein quartic modulo 3 is extremal over 
fields of the form Fgm . If to is odd, its number of points is maximal. If to is even, its 
number of points is minimal. Although the curve C3 is not isomorphic over F3 to 
the Klein quartic, over F27 it has the same characteristic polynomial of Frobenius, 
being (T 2 + 27) 3 . It follows that C3 shares the extremal properties of the Klein 
quartic over fields of the form F 3 e m : C3 has the maximal number of points possible 
if to is odd, and the minimal number of points possible if m is even. 

We conclude with the following question: Is there a hyperelliptic curve (of any 
genus) defined over a finite field which is not nondegenerate? If so, it might also 
have interesting extremal properties. 
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